We recover a video of the motion taking place in a hidden scene by observing changes in indirect illumination in a nearby uncalibrated visible region. We solve this problem by factoring the observed video into a matrix product between the unknown hidden scene video and an unknown light transport matrix. This task is extremely ill-posed, as any non-negative factorization will satisfy the data. Inspired by recent work on the Deep Image Prior, we parameterize the factor matrices using randomly initialized convolutional neural networks trained in a one-off manner, and show that this results in decompositions that reflect the true motion in the hidden scene.

In this paper, the authors study the problem of reconstructing a hidden scene from the observed videos. The proposed method seeks to invert tight transport matrix without a calibration step. The problem is challenging and ill posed. The author learn a low-dimensional basis from observed videos and use deep image prior models for generating hidden scene and coefficients of the light transport basis. Originality: The paper uses inverse light transport to recover a video of hidden scene without any calibration, which seems novel.

This paper has tackled an extremely challenging problem. It provides a neat and bold idea towards solving it. While the work is far from complete, as agreed upon by many of the reviewers in a discussion, it provides a first-cut idea and attempt, and enough detail to potentially carry this proof-of-concept further in the future. I suggest the authors carefully address the reviewer's comments.

We recover a video of the motion taking place in a hidden scene by observing changes in indirect illumination in a nearby uncalibrated visible region. We solve this problem by factoring the observed video into a matrix product between the unknown hidden scene video and an unknown light transport matrix. This task is extremely ill-posed, as any non-negative factorization will satisfy the data. Inspired by recent work on the Deep Image Prior, we parameterize the factor matrices using randomly initialized convolutional neural networks trained in a one-off manner, and show that this results in decompositions that reflect the true motion in the hidden scene.

We recover a video of the motion taking place in a hidden scene by observing changes in indirect illumination in a nearby uncalibrated visible region. We solve this problem by factoring the observed video into a matrix product between the unknown hidden scene video and an unknown light transport matrix. This task is extremely ill-posed, as any non-negative factorization will satisfy the data. Inspired by recent work on the Deep Image Prior, we parameterize the factor matrices using randomly initialized convolutional neural networks trained in a one-off manner, and show that this results in decompositions that reflect the true motion in the hidden scene. Papers published at the Neural Information Processing Systems Conference.